Fox News senior White House correspondent Jacqui Heinrich speaks with White House National Security Communications Adviser John Kirby about the mysterious drones in the Garden State on'The Story.' A U.S. Senator from Connecticut said the mysterious drones spotted recently flying over states in the mid-Atlantic region should be "shot down, if necessary." In an interview on Capitol Hill Thursday, Sen. Richard Blumenthal, D-Conn., said intelligence analysis is needed on the drones and the U.S. must act "more aggressively" against the sightings that have been reported. "We should be doing some very smart intelligence analysis and take them out of the skies, especially if they're flying over airports or military bases," the senator said. "They should be shot down, if necessary, because they're flying over sensitive areas."

Building on their demonstrated ability to perform a variety of tasks, we investigate the application of large language models (LLMs) to enhance in-depth analytical reasoning within the context of intelligence analysis. Intelligence analysts typically work with massive dossiers to draw connections between seemingly unrelated entities, and uncover adversaries' plans and motives. We explore if and how LLMs can be helpful to analysts for this task and develop an architecture to augment the capabilities of an LLM with a memory module called dynamic evidence trees (DETs) to develop and track multiple investigation threads. Through extensive experiments on multiple datasets, we highlight how LLMs, as-is, are still inadequate to support intelligence analysts and offer recommendations to improve LLMs for such intricate reasoning applications.

In this project, we test the effectiveness of Large Language Models (LLMs) on the Abstraction and Reasoning Corpus (ARC) dataset. This dataset serves as a representative benchmark for testing abstract reasoning abilities, requiring a fundamental understanding of key concepts such as object identification, basic counting, and elementary geometric principles. Tasks from this dataset are converted into a prompt-based format for evaluation. Initially, we assess the models' potential through a Zero-shot approach. Subsequently, we investigate the application of the Chain-of-Thought (CoT) technique, aiming to determine its role in improving model performance. Our results suggest that, despite the high expectations placed on contemporary LLMs, these models still struggle in non-linguistic domains, even when dealing with simpler subsets of the ARC dataset. Our study is the first to concentrate on the capabilities of open-source models in this context. The code, dataset, and prompts supporting this project's findings can be found in our GitHub repository, accessible at: https://github.com/Lianga2000/LLMsOnARC.

News summary generation is an important task in the field of intelligence analysis, which can provide accurate and comprehensive information to help people better understand and respond to complex real-world events. However, traditional news summary generation methods face some challenges, which are limited by the model itself and the amount of training data, as well as the influence of text noise, making it difficult to generate reliable information accurately. In this paper, we propose a new paradigm for news summary generation using LLM with powerful natural language understanding and generative capabilities. We use LLM to extract multiple structured event patterns from the events contained in news paragraphs, evolve the event pattern population with genetic algorithm, and select the most adaptive event pattern to input into the LLM to generate news summaries. A News Summary Generator (NSG) is designed to select and evolve the event pattern populations and generate news summaries. The experimental results show that the news summary generator is able to generate accurate and reliable news summaries with some generalization ability.

Artificial Intelligence is an emerging field in which humans are making machines that are capable of making decisions on their own. These machines or robots integrate information, analyze critical data, and make decisions on the basis of given information. It is a very advanced technology as robots are doing daily tasks like human beings. But there is always a thing that is missing, common sense. Humans are making robots more accurate and making them able to make decisions with more precision.

We present a model of sense-making that greatly facilitates the collaboration between an intelligent analyst and a knowledge-based agent. It is a general model grounded in the science of evidence and the scientific method of hypothesis generation and testing, where sense-making hypotheses that explain an observation are generated, relevant evidence is then discovered, and the hypotheses are tested based on the discovered evidence. We illustrate how the model enables an analyst to directly instruct the agent to understand situations involving the possible production of weapons (e.g., chemical warfare agents) and how the agent becomes increasingly more competent in understanding other situations from that domain (e.g., possible production of centrifuge-enriched uranium or of stealth fighter aircraft).

Argumentation provides a representation of arguments and attacks between these arguments. Argumentation can be used to represent a reasoning process over evidence to reach conclusions. Within such a reasoning process, understanding the value of information can improve the quality of decision making based on the output of the reasoning process. The value of an item of information is inherently dependent on the available evidence and the question being answered by the reasoning. In this paper we introduce a value of information on argument frameworks to identify the most valuable arguments within the finite set of arguments in the framework, and the arguments and attacks which could be added to change the output of an evaluation. We demonstrate the value of information within an argument framework representing an intelligence analysis in the maritime domain. Understanding the value of information in an intelligence analysis will allow analysts to balance the value against the costs and risks of collection, to effectively request further collection of intelligence to increase the confidence in the analysis of hypotheses.

Analytic software tools and workflows are increasing in capability, complexity, number, and scale, and the integrity of our workflows is as important as ever. Specifically, we must be able to inspect the process of analytic workflows to assess (1) confidence of the conclusions, (2) risks and biases of the operations involved, (3) sensitivity of the conclusions to sources and agents, (4) impact and pertinence of various sources and agents, and (5) diversity of the sources that support the conclusions. We present an approach that tracks agents' provenance with PROV-O in conjunction with agents' appraisals and evidence links (expressed in our novel DIVE ontology). Together, PROV-O and DIVE enable dynamic propagation of confidence and counter-factual refutation to improve human-machine trust and analytic integrity. We demonstrate representative software developed for user interaction with that provenance, and discuss key needs for organizations adopting such approaches. We demonstrate all of these assessments in a multi-agent analysis scenario, using an interactive web-based information validation UI.

There is an extraordinary amount of data being generated around the world on a daily basis. The task of collecting relevant information, organizing it, and piecing it together in a way that tells a story seems like an overwhelming and nearly impossible task. Yet, this is the monumental task of intelligence analysts. Thankfully, incredible technological advancements, including machine learning (ML) and artificial intelligence (AI), are assisting analysts in their efforts to collect and categorize massive amounts of data. Technology is evolving at a rapid pace, and analysts must always be learning how to apply machine-learning technology to help them better understand and solve complex problems.

There is a joke spies like to tell. They say prostitution is the world's oldest profession, and espionage is the second. Now, that self-proclaimed second-oldest profession is facing a seismic shift: Artificial intelligence is pervading to the intelligence community. American intelligence isn't what it used to be. Some of the types of secrets are changing, morphing from hidden gems cemented behind walls of government hush-hush to emerging signals washed out by open-source noise.